You delete files as you do with regular files in the Windows Explorer.
You can delete a file by selecting the Delete menu-item in the File menu or pressing the Delete key. You can delete multiple files by high-lighting multiple entries in the Explorer view.
You may also delete files by dragging them to the Windows Recycle Bin. The files will however not be put inside the Recycle Bin, so you cannot undo the delete action.
GMail Drive does not support Rename of files.